Network - Backend

  • <excalidraw_not_loaded>

Network Architecture

  • Types of Multiplayer .

    • WebServer instead of a DedicatedServer, for simple things.

      • Example: pinball game with scoreboard.

    • Ghost Data instead of Concurrent Multiplayer.

      • Ghost Data refers to just a REPLAY of a different player, requiring only 1 network transfer at the start of the level, instead of being real-time.

      • Supposedly widely used in racing games and mobile games.

      • Example: penguin game.

    • Tips to make lag less frustrating:

      • Players can walk through other players.

        • In case of lag, there will be no collision problems.

      • It doesn’t matter who kills the enemy, everyone gets credit, no need to fight over loot.

      • "Basically single-player++".

  • Interview with Pat Wyatt .

    • Very good video.

    • Talks about random network things, with little focus on Netcode.

    • Overall, it's basically an interview with someone who works on networking and the challenges in each game.

    • Worth watching from start to finish.

    • Warcraft (1994):

      • Uses Lockstep, since the game is an RTS.

      • It's said that the code was not clean; it’s a mess.

    • Diablo (1997):

      • The game was made with a different network architecture in mind, so it was messy initially.

      • Determinism is required for lockstep.

      • The game was written in Haste.

      • P2P Lockstep, Client Authority was implemented.

      • {16:25}

        • "Does the client immediately see something happening, or wait for server confirmation?"

          • Answer: "You kill on your screen and inform the Level Master that the creature died, very loose. Easy to cheat."

          • Very insecure. Client-authority.

      • "Don't do a game like this. People will cheat".

    • Starcraft (1998):

      • Same as previous games.

    • Guild Wars (2005):

      • {19:42} File serving:

        • "Downloads the game while playing."

        • Everything is cached on the drive.

        • Each file has an ID and version.

        • "delta compress the difference between the file a person has and a file a person should have".

        • TCP.

      • {36:00}

        • Everything uses TCP throughout the game.

        • The host for the game is chosen.

        • Server separation:

          • "Component isolation split it into several servers."

          • "Even if the servers run on the same machine, it's better to have separate servers for organization and crash understanding."

          • Bottlenecks are discussed, so splitting into smaller servers allows horizontal scaling by adding more machines instead of upgrading the same one.

          • Also provides uptime protection: if one machine fails, others keep the connection active.

        • Protobuf is praised.

        • The game rarely needed downtime for updates.

        • Discusses TCP vs UDP extensively.

          • UDP is good for FirstPersonShooters.

          • TCP can be problematic with errors, mobile connections, etc.

        • "The server is authoritative for important things like position and which items players have, but clients..."

        • Tips:

          • "Always code assuming 1-minute RTT (ping) to consider failure cases."

Authority

  • It's a dilemma between synchronizing actions or synchronizing the consequences of actions.

Server-Authority

  • "The only 'authority' the client has is regarding its inputs, which are sent to the server, which 'notifies' the other clients."

  • "You want the client to have as little 'authority' as possible, so the server is the source of truth."

  • Inputs are done on the Client and passed to the Server, and ONLY the Server can move Entities and world objects.

  • The Client player is just an amoeba copying the Server's movements and properties, with no agency.

  • The Server has access to all "processes," while peers only have access to Inputs.

  • Synchronizing everything can be cumbersome and may cause unresponsiveness during lag.

  • Lag compensation is often necessary.

  • Safer against cheating.

  • Demonstration of this method ~{26:00 -> end} .

Client-Authority

  • The Client really PLAYS on the Client, while using RPCs to make other Clients' entities execute the same actions.

Peer-to-Peer (P2P)

  • Clients communicate directly with each other, without a central server making decisions.

  • Host-migration :

    • When the host disconnects, another host is chosen from the remaining peers.

    • Used in Halo, causing 'black screens' while waiting for the new host, and potential inconsistencies.

  • Hybrid 'P2P-Dedicated Server' in Destiny 1 .

    • Mainly discusses why P2P was chosen to ensure a 'responsive singleplayer experience' with better connectivity via ping-based matchmaking.

    • To compensate for P2P Host Migration issues, some crucial game states are saved on the Dedicated Server.

    • .

      • Hosts act as 'physics host,' simulating behavior, while critical mission progression is handled via the cloud.

      • Different 'host bubbles' can interact in the same activity, each with its own physics simulation, but mission progression remains the same.

    • Security issues of P2P are not discussed.

      • Exploit bugs are shown where raid bosses die in 1 hit because the host unplugged the LAN cable.

      • "We do allow a certain amount of exploit to happen in an individual console. We rely on other metrics to monitor and ban the player, retroactively."

    • "the host uses 44mb of data when they're running."

Dedicated Server

  • if OS.has_feature('dedicated_server'): .

  • if DisplayServer.get_name() == "headless": .

  • if "--server" in OS.get_cmdline_user_args(): .

  • game.exe --server --headless .

  • Exporting to a dedicated server .

  • The 'headless' version of the game should be lightweight, as it doesn’t need most scenes or visual elements, running in the terminal.

  • Server creation, with upload to Amazon EC2 .

    • "Amazon EC2 (Elastic Compute Cloud) is a cloud computing service by AWS, allowing users to rent virtual machines to run applications, websites, databases, and other services. With EC2, you can quickly launch and configure virtual servers, adjusting computing capacity to project needs, and pay only for the resources you use."

  • "Client on Windows, Server on Linux, much cheaper."

  • "Serverless":

    • Only works when someone is playing.

Bottlenecks

Godot High-level Multiplayer

Explanations
Limitations of Godot's ENet Implementation

  • Although ENet is an excellent low-level multiplayer networking library, Godot implements ENetMultiplayerPeer  in a simplified way, which may not be the best choice for large-scale games like MMOs  due to lack of flexibility in network control, scalability, and customization.

  • Lack of clustering and load distribution support :

    • ENetMultiplayerPeer does not have built-in resources for load balancing between servers or for dynamic player partitioning.

    • Network systems for MMOs usually use a distributed network structure so high-density player areas don’t overload a single server. This is not natively supported in ENet.

  • Persistence and Global State :

    • MMOs generally require maintaining a persistent global state (world information, inventory, player progression, etc.). While ENet provides reliable and ordered communication, it doesn’t handle persistence or state replication.

    • For MMOs, it is common to use a network structure that interacts with databases and robust caching systems. Custom solutions for this kind of persistence are not native to ENet.

  • Security and Moderation :

    • ENet offers little native security against common attacks on MMO servers, such as DDoS or packet manipulation. Protocols developed for MMOs typically implement robust authentication, encryption, and attack mitigation mechanisms, often with additional support for cheat detection.

High-level API

MultiplayerAPI

  • Abstract implementation. Cannot be instantiated.

SceneMultiplayer

  • Concrete implementation of MultiplayerAPI, can be instantiated.

MultiplayerAPIExtended

  • get_tree().set_multiplayer(MultiplayerExtended.new())

    • The pathing is not passed, since I want to override the default multiplayer.

    • By doing this, multiplayer == _game_server == get_tree().get_multiplayer()  for ANY node in the game.

      • This is valid even for nodes that spawned later.

    • This can be done in _enter_tree() , _ready() , or elsewhere, but prefer _enter_tree() , since the earlier the better.

  • Using MultiplayerAPIExtension causes multiplayer.get_remote_sender_id()  to not work.

Synchronization

Authority

  • Authority cannot be set in _ready() .

MultiplayerSynchronizer

  • Serialization :

    • "Also serializes automatically".

  • Synchronization of spawned objects :

    • If MpSync misses the first sync rotation in the scene, it will not receive further signals.

    • Changing visibility to PubliclyVisible outside of _ready()  does nothing, you must use set_visibility(peer_id)  or a visibility_filter to apply visibility changes.

MultiplayerSpawner

  • Synchronization of spawned objects :

    • Spawned objects will copy everything defined between .instantiate()  and add_child() . After add_child() , no more copying occurs.

    • If a client logs into the server after something has been spawned by MpSpawner, the new client will also get these spawns, even if they were not present at the time of the spawn on the Server.

    • Only use _death.rpc()  if you are not using MpSpawner, since MpSpawner WANTS to despawn all instances of the instantiated scene.

    • (2024-11-11) Dynamic spawns do not work well with my CustomMpSync script.

RPC (Remote Procedure Calls)

  • "request / fetch -> receive / retrieve."

  • Functions callable on other peers.

  • RPC or MultiplayerSynchronizer for inputs?

    • Via MultiplayerSynchronizer:

      • input_jump = Input.get_action_strength(&'jump') .

  • Serialization :

    • "If I pass a parameter to an RPC call, is it automatically serialized before sending it to the other peers?"

      • Serialization and deserialization is automatic.

  • Cannot send :

    • Objects.

      • Includes Nodes and Resources.

        • Demonstration of what 'not serialize' looks like .

        • "Is there a reason for RPC calls to not serialize objects (nodes and resources)? Seems to be a way to serialize the object, so I wonder why it isn't used for RPC calls."

          • Objects in Godot have been designed with the ability to contain their own scripts if needed.

          • Which unfortunately means that if an RPC could carry an object, it could also carry a custom malicious script that would run _init  automatically on the other PC and basically turn Godot into a trojan.

          • Also, if you try to send an object, it will convert it to an object ID, which is meaningless on the other PC.

    • Callables.

      • Other peers only receive null  in place of the Callable.

  • Errors :

    • When calling an RPC on the Server, but the Client does not have the node:

      • The Server has node authority; i.e., authority was not changed.

  • Quirks :

    • Same Node Path:

      • For a remote call to be successful, the sending and receiving node need to have the same NodePath , which means they must have the same name.

        • If an RPC resides in a script attached to /root/Main/Node1 , then it must reside in precisely the same path and node on both the client script and the server script.

      • When using add_child()  for nodes expected to use RPCs, set the argument force_readable_name  to true .

    • Declaration on Server and Client for dedicated servers:

      • If a function is annotated with @rpc  on the client script (resp. server script), then this function must also be declared on the server script (resp. client script). Both RPCs must have the same signature which is evaluated with a checksum of all RPCs . All RPCs in a script are checked at once, and all RPCs must be declared on both client and server scripts, even functions that are currently not in use .

  • Points ok :

    • Calling the RPC in the case of a Dedicated Server can be done with either callable.rpc_id(peer_id)  or rpc_id(peer_id, &'callable_name') . The first option is better and safer as it does not involve strings.

    • (I think this is false) Function arguments can be different.

      • Function arguments are not checked for matching between server and client code (example: func sendstuff():  and func sendstuff(arg1, arg2):   will pass  signature matching).

  • Peer id :

    • "The one calling the function is peer 0 and ALWAYS exists, otherwise they could not call the function to begin with. Peer ids are dependent on the perspective of the client.  If you have client A and client B connected to each other. Client A will see client B as peer 1 and vice versa. And each of those will see themselves as peer 0."

Others
Ensuring Connection

  • Port-forwarding

    • If you're hosting a server on your own machine and want non-LAN clients to connect to it, you'll probably have to forward the server port on your router. This is required to make your server reachable from the Internet since most residential connections use NAT.

    • My Public IP .

  • Android

    • When exporting to Android, make sure to enable the INTERNET  permission in the Android export preset before exporting the project or using one-click deploy. Otherwise, network communication of any kind will be blocked by Android.

Channels

  • For example, game chat messages and some core gameplay messages should all be sent reliably, but a gameplay message should not wait for a chat message to be acknowledged. This can be achieved by using different channels.

  • Modern networking protocols support channels, which are separate streams within the connection. This allows for multiple packet streams that do not interfere with each other.

Security and Authentication

Hash Tables

  • Explanation of Hash Tables .

    • "The memory location where the value will go (index) is determined by the value of the object itself."

    • Great video.

  • Conflict resolution :

    • Best case (no collisions):

      • $O(1)$.

    • 'Open addressing' techniques:

      • Linear Probing.

        • Worst case:

          • $O(n)$.

      • Quadratic Probing.

        • Worst case:

          • $O(?)$.

    • 'Closed addressing' techniques:

      • Linked List (Chained List).

Token Authentication

  • My interpretation :

    • The Token is the access key for the connection between the Client and the GameServer. The Client verifies its credentials with the GatewayServer -> AuthenticationServer just to obtain a Token that will be accepted by the GameServer.

    • Upon successful authentication, the Client and the GameServer receive the same Token, so the connection can only be established between the two if there is a match.

    • This extra security layer is for the case the Client tries to connect to the GameServer without confirming its credential.

      • Supposedly this could happen if the Client manages to call the connection functions to the GameServer via "recompile" of the code, who knows.

  • Allows players to stay connected without sending their credentials (username and password) for every action in the game.

  • Login flow:

    • The player logs in with credentials. The server validates the information.

    • After successful verification, the server issues a token (like a JWT - JSON Web Token), representing the player's identity and some permissions.

    • This token is sent to the client (the player's device) and temporarily stored (in memory or local storage).

    • For each interaction with the server (requesting game progress, buying items, etc.), the client sends the token instead of the credentials.

  • Advantages:

    • Session and Logout Control:

      • Token expiration also allows stricter session control. If the player disconnects from a session or changes devices, an expired token can prevent the player from being considered "logged in" in multiple locations simultaneously, which can cause conflicts in online games.

      • This allows developers to "end" an active session in case of suspicious activity or in response to a player logout request.

  • Duration:

    • In games, tokens commonly last between 1 to 12 hours, allowing the player to have a continuous session without needing to re-login. This provides security without compromising user experience.

  • Alternatives to Token Authentication:

    • Login Session with Cookies:

      • Common in websites and some web apps, where the server stores each user's session and identifies them via cookies.

      • In online games, especially MMOs, maintaining sessions at scale can be complicated and resource-intensive, making it less scalable than token authentication.

    • Constant Credential Resending:

      • The client would send credentials (username and password) for each new request.

      • This exposes credentials to more interception attacks and is insecure, especially on unstable networks. Each new request represents a risk of credential capture, increasing system vulnerability.

    • Certificate-based Authentication:

      • In some critical applications, digital certificates are used for mutual authentication, where both the client and server have private keys.

      • It is secure, but very complex to implement in games and difficult to manage for each player, especially in amateur or smaller-scale games.

  • Token creation and verification .

    • Random number -> convert to String with sha256 -> unix_time as String -> concatenate the two strings (just put one in front of the other).

      • The 'unix_time' is based on the user's computer date, which may be questionable.

    • {~13:50}

      • Token verification process, via comparing the current time and the clock concatenated at the end of the Token; if greater than 30 (seconds), the Token is removed from the "expected Tokens" list.

    • Once the Token is used to login, it is removed from the 'expected Tokens' list.

Authentication Server

  • In GameDev:

    • Defined in a separate project.

  • Responsible for verifying credentials and issuing tokens. In games, it acts as the "security gatekeeper."

  • Login process (without the Gateway):

    • The player opens the game and sends credentials to the authentication server.

    • The server checks the password hash and salt in the database to confirm the player's identity.

    • If everything is correct, the authentication server generates a token and returns it to the client.

    • This token is used in subsequent client communications with the game server.

Gateway

  • In GameDev:

    • Defined in a separate project.

  • Acts as an intermediary between the client (player's device) and the game servers. It acts as a "gate," facilitating communication and protecting main servers from direct access, while providing centralized traffic and security control.

  • Purposes

    • Security against attacks:

      • The gateway is the first point of contact for all client requests. This allows it to filter suspicious or malicious traffic before it reaches main game servers.

      • It can block common attacks like DDoS (many requests sent to overload the system) and code injection attacks (hackers inserting malicious code).

      • In some systems, it also checks authentication tokens, ensuring only authenticated players access game features.

    • Authentication Server Security:

      • The gateway works with the authentication server: it receives initial login requests and forwards them to the authentication server. Once the player is authenticated, the gateway can redirect the request to the correct game server.

      • This adds an extra protection layer since the authentication server is never directly exposed to clients.

    • Encryption:

      • The gateway is responsible for implementing and managing connection encryption. It ensures all communication between client and servers is secure, using protocols like HTTPS or TLS (Transport Layer Security).

      • This means that even if an attacker intercepts the communication, they cannot read the data due to encryption.

    • Load Balancing

      • As online games attract many players simultaneously, the gateway can act as a load balancer, distributing requests across servers evenly, avoiding overload.

      • The gateway ensures all player connections flow smoothly and the system continues to work even under heavy load.

    • Monitoring:

      • As the gateway receives all player requests, it serves as a central point to monitor and log traffic. Useful for developers to analyze usage patterns, detect errors, and suspicious activity.

      • It also allows the security team to quickly identify unusual behaviors, like login spikes or unusual activity, and take preventive measures.

  • Login process:

    • The player opens the game and enters credentials.

    • The client (game app) sends the login request to the gateway.

    • The gateway redirects the request to the authentication server, which verifies credentials.

    • After authentication, the server issues a token and the gateway sends it back to the client.

    • With the token, the client sends a new request (e.g., to start a match) to the gateway.

    • The gateway verifies the token and, if valid, redirects the request to the correct game server.

  • "Initially, a Gateway may not be necessary. For a small MMO, you can manage traffic directly with the game server and authentication server. If the game grows and requires multiple servers, the gateway becomes very useful for load balancing and security."

Local Network

  • Yes, you can connect the GameServer , GatewayServer , and AuthenticationServer  all on the same local network, and this is common practice for MMOs and other online games requiring fast and secure communication between different server components. This setup benefits from reduced latency, security, and simplicity.

  • Recommended to do so. There are significant advantages to managing everything within the same internal network. It's safer.

  • Complications:

    • Local Network Bottleneck :

      • Depending on the number of players, traffic may overload the local network. Ensure the infrastructure (routers and switches) supports the required load.

    • Single Point of Failure Dependency :

      • With all servers on the same network, a failure in the local network infrastructure can affect all of them. Redundancy for the network and backups for each server can mitigate this.

  • Keeping GameServer , GatewayServer , and AuthenticationServer  on a single machine can be viable, especially considering an initial number of 10 players.

TLS (Transport Layer Security), SSL (Secure Sockets Layer) and DTLS

TLS

  • SSL is a security protocol designed to ensure secure communication on the internet, mainly between browsers and servers.

  • It creates an encrypted connection that protects transmitted data, preventing third parties from intercepting or altering the information.

  • When you access a website using SSL (indicated by "https://" in the URL), the server presents an SSL certificate issued by a trusted certificate authority (such as VeriSign, DigiCert). The browser verifies the validity of this certificate before establishing the connection.

  • In addition to encrypting and authenticating, SSL also ensures that data has not been altered or corrupted during transit. It uses functions called hashes to verify data integrity.

  • A website using SSL displays a padlock in the browser and uses "https://" instead of "http://".

  • Characteristics :

    • TLS uses TCP:

      • TCP is a connection-oriented protocol, ensuring the order and delivery of all packets. If a packet is lost, TCP automatically attempts to retransmit it and puts the packets in the correct order for the application.

TLS vs SSL

  • The most current and secure version of the protocol is TLS, which is an evolution of SSL and provides greater security and efficiency.

  • Nowadays, most secure communications on the internet use TLS, but the term "SSL" continues to be used as a general reference.

DTLS (Datagram Transport Layer Security)

  • It is a version of the TLS (Transport Layer Security) protocol, but for UDP (User Datagram Protocol).

  • It is mainly used in network applications that require secure communication, but where speed and low latency are priorities, such as voice and video calls, online games , and streaming.

  • Main Features of DTLS :

    • Low-latency security : DTLS allows fast and secure communication without the delivery and ordering guarantees of TCP, which is essential for real-time applications.

    • Connection independence : Since UDP does not establish a formal "connection," DTLS must also handle sessions securely even without the guaranteed connection steps of TCP.

    • Optional retransmission support : DTLS tries to retransmit important data (such as parts of the initial handshake) if packet loss compromises the authentication process, but this retransmission is limited since UDP's priority is to minimize latency.

  • Generating an X509 Certificate for the DTLS Security Protocol in Godot .

    • A .crt file (certificate) and a .key file (encryption key) are generated.

    • The certificate is created and used during the Client-to-Server connection process.

  • Usage in Godot 4 with ENetMultiplayerPeer :

    • Server: 

    const _certificado_x509 : X509Certificate = preload('res://certificados/X509_certificado.crt')
const _key : CryptoKey = preload('res://certificados/X509_key.key')

func _criar_servidor_gateway() -> Error:
    var peer := ENetMultiplayerPeer.new()
    var erro : Error = peer.create_server(_PORTA_GATEWAY, _CONEXOES_MAXIMAS)
    if erro:
        return erro
        
    peer.get_host().dtls_server_setup(TLSOptions.server(_key, _certificado_x509))
        
    get_tree().set_multiplayer(_servidor_gateway, get_path())
    _servidor_gateway.set_multiplayer_peer(peer)

    return OK

    • Client: 

    const _certificado_x509 : X509Certificate = preload('res://certificados/X509_certificado.crt')

func entrar_no_servidor_gateway() -> Error: 
    var peer := ENetMultiplayerPeer.new()
    var erro := peer.create_client(_SERVER_IP_DEFAULT, _PORTA_GATEWAY)
    if erro:
        return erro
        
    peer.get_host().dtls_client_setup(_SERVER_IP_DEFAULT, TLSOptions.client_unsafe(_certificado_x509))
        
    get_tree().set_multiplayer(_servidor_gateway, get_path())
    _servidor_gateway.set_multiplayer_peer(peer)
        
    return OK

    • Tested and works 100%.

      • If you use TLSOptions.client()  instead of TLSOptions.client_unsafe() , errors occur and the connection is not established, as expected.

      • If the certificate differs between Client and Server, errors occur and the connection is not established, as expected.

X.509 Certificate

  • An X.509 certificate is a requirement for TLS/SSL protocols.

  • It is a type of digital certificate that follows a widely used standard to authenticate and establish secure connections in communication systems, such as in Public Key Infrastructure (PKI). It is crucial to ensure data security and the authenticity of online identities, especially in protocols like TLS/SSL , used for secure HTTPS connections.

  • The relationship between X.509 and DTLS lies in the USE  of digital certificates (X.509) as part of the authentication and encryption process in secure connections established with DTLS.

.crt File

  • A .crt file is a type of file that contains a digital certificate, usually in the X.509  format.

OAuth (Open Authorization)

OAuth vs Auth0

  • OAuth (Open Authorization) is an authorization protocol that allows you to grant access to protected resources on a server (for example, accessing information from a Google or Facebook account) without sharing your credentials (such as username and password). It is widely used to delegate permissions securely between different systems.

    • Example: When using Google or Facebook login on a website, OAuth enables this authentication.

  • Auth0 is an identity-as-a-service platform that provides complete authentication and authorization solutions. It can be used to implement protocols like OAuth , OpenID Connect, SAML, among others, but Auth0 is a complete solution  that abstracts and simplifies authentication and authorization implementation.

Communication Protocols

RPC (Remote Procedure Calls)

  • RPC is a protocol that allows a program to execute functions on another computer or remote server as if it were calling a local function. It abstracts the complexity of network communication, making it appear as though the program is calling an internal function, when in fact the function resides on another system.

gRPC (Google RPC)

  • Uses Protobuf  (Protocol Buffers) for data serialization and HTTP/2  for transport.

  • Structure :

    • Each gRPC message starts with a 5-byte length field.

      • The first byte is used to store a flag, while the next 4 bytes represent the total message size.

    • HTTP/2 divides communication into frames.

      • Each frame carries a payload size, and the frames are used to transport the actual message, being reassembled at the destination to reconstruct the original message.

  • Widely used in distributed systems and microservices.

JSON-RPC

  • An RPC protocol that uses the JSON format for requests and responses, generally used for web systems.

XML-RPC

  • Similar to JSON-RPC but uses XML to structure requests and responses.

HTTP

WebSockets

  • WebSockets are a communication protocol that allows a bidirectional , real-time  connection between a client (usually a browser) and a server.

    • If these two features are not needed, HTTP may be better due to its simplicity.

  • Unlike HTTP, which is based on a request/response  model (the client makes a request and the server responds), the WebSocket maintains an open and persistent connection between client and server, allowing both to send and receive data at any time.

    • It is based on signals/events.

  • It still uses a TCP  connection, which is good for reliability but not for latency, so it is not ideal for real-time applications like VoIP and fast-paced games.

Usage Examples

  • Real-time chats.

  • Multiplayer games.

  • Financial trading applications.

Address (URI)

  • ws://  (Unsecured WebSocket):

    • The standard, unencrypted WebSocket protocol.

    • Used for communication on local networks or when security is not a priority, such as in development environments or internal networks.

    • Works similarly to http:// , without encryption or a security layer.

  • wss://  (Secure WebSocket):

    • The secure version of WebSocket, where communication is encrypted using TLS (Transport Layer Security), the same security mechanism used by https:// .

    • Used when security is important, such as in production environments where sensitive data is transmitted.

    • Ensures that communication is protected against interception and attacks (such as man-in-the-middle attacks).

Explanations

In Godot

  • WebSockets .

  • Using WebSockets to Turn an Android into a Controller .

    • The example basically follows Godot's documentation.

    • Two different projects are used, one to run the game and another to control the game from Android.

    • On Android, the App acts as a controller for the game running on the PC.

    • The Android device uses the accelerometer to control a ship on the PC.

    • Communication is done via sending and receiving packets, without @rpc .

In Python

Requirements

  • pip install websockets .

  • Use of 'asyncio'.

Example 
import asyncio
import websockets

async def echo(websocket, path):
    async for message in websocket:
        print(f"Received message: {message}")
        # Echo the message back to the client
        await websocket.send(f"Echo: {message}")

start_server = websockets.serve(echo, "0.0.0.0", <port>)

asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

  • About asynchrony:

    • asyncio  is Python's standard library for asynchronous programming.

    • Using asynchronous functions is essential for creating responsive, efficient, and scalable applications, especially in environments where I/O and network communication are common. They allow cleaner, easier-to-maintain code and optimize system resource usage. If you are developing an application involving operations that may take time, using asynchronous functions is a good practice.

FTP (File Transfer Protocol)

  • Communication protocol used to transfer files between a client and a server over a network.

  • Not secure by default (uses plaintext), but can be combined with SSL/TLS (FTPS) to add security.

Communication Protocols: Low-level

Sockets

  • Each socket = one endpoint of a single connection.

Socket IDs (File Descriptors) (fd)

  • File descriptor :

    • The term file descriptor comes from Unix and Unix-like operating systems, and it’s deeply rooted in how these systems abstract I/O.

    • In Unix philosophy, "everything is a file".

      • Disk files

      • Pipes

      • Devices

      • Sockets

      • Terminals

      • Etc.

    • So:

      • When you open a file → you get a file descriptor.

      • When you open a socket → you also get a file descriptor.

    • They’re both handled through the same I/O API: read() , write() , close() , etc.

    • This abstraction is why sockets use file descriptors — they’re just "special files" from the OS’s point of view.

  • Usage :

    • The socket ID is used by your program to interact with the OS-managed socket.

    • The OS routes TCP packets based on the socket's internal connection tuple, not your program directly.

      • Local Socket A: 127.0.0.1:6060  

      • Local Socket B: 127.0.0.1:50543      ->     127.0.0.1:6061

    • The OS  stores and manages the mapping between:

      • Socket ID

      • The internal state of that connection

      • File/socket types, flags, permissions, etc.

    • You can think of the socket ID  as an index or key into a kernel-managed table of open resources, much like a hashmap, though it's typically implemented as a simple indexed array inside the OS.

  • You can safely have one listening socket on 6060  and one connected socket to 6061  in the same program.

TCP (Transmission Control Protocol)

  • A reliable communication protocol that guarantees the delivery of data packets between two devices.

  • It is part of the TCP/IP  suite and is used by many protocols such as HTTP, HTTPS, FTP, etc.

  • Godot:

    • "TCP ensures packets will always arrive reliably and in order, but latency is generally higher due to error correction. It's also quite a complex protocol because it understands what a "connection" is, and optimizes for goals that often don't suit applications like multiplayer games. Packets are buffered to be sent in larger batches, trading less per-packet overhead for higher latency. This can be useful for things like HTTP, but generally not for games. Some of this can be configured and disabled (e.g. by disabling "Nagle's algorithm" for the TCP connection)."

Protocol

  • SYN -> SYN ACK -> ACK.

  • .

  • TCP packets do not contain a reliable sender ID that can be verified directly.

Closing ( close )

  • What it does :

    • The socket's file descriptor (e.g., sockfd ) is marked as available for reuse.

    • Decrements the reference count for the socket.

      • Sockets may have multiple references (e.g., via dup() , fork() , or threading).

      • Resources are only freed (e.g., memory, kernel structures) when the reference count reaches zero.

    • It initiates connection teardown (sends FIN  if it’s the last reference).

  • The integer value representing the descriptor can now be reassigned by future socket() , open() , or similar calls.

  • The client cannot close the connection to the server; it's impossible. Reasons:

    • The client doesn't have the server socket available to it.

      • The client has no way to reference the server’s socket.

    • While TCP connections are bidirectional (peer-to-peer), socket descriptors are local to each process.

      • The client and server communicate via IP:Port pairs, but their socket file descriptors ( sockfd ) are private.

      • The kernel enforces this separation for security and stability.

    • Even if the client sends malicious packets (e.g., a spoofed RST ), the server’s OS will handle it at the TCP layer, not the application socket layer.

  • The only way  the server’s socket closes is if:

    1. The server explicitly calls close()  on its socket.

    2. The server process crashes (kernel cleans up resources).

    3. The OS terminates the process (e.g., kill -9 ).

Shutdown ( shutdown )

  • What it does :

    • Forces a partial or full closure of communication in a specified direction.

    • Does not free the socket descriptor (still needs close()  later).

    • Immediately affects the TCP connection state (sends FIN  or RST ).

Connection

  • Server :

    • listen

      • Puts the socket into passive mode, indicating it will accept connections.

      • “I am ready to accept TCP connections.”

      • Necessary before: accept .

      • Relevant only for TCP (stream) sockets, not UDP.

      • Internally does a bind .

    • bind

      • Associates the socket with a local address (IP + port).

      • “I want to use this IP:port for this socket.”

    • accept

      • In Odin , the accept()  call on a TCP socket is blocking by default, meaning execution halts until a client tries to connect to the socket.

        • Blocks until a client does dial()  to that address/port.

        • Non-blocking mode :

          • You need to configure the socket with system flags, usually requiring a direct OS syscall (not abstracted by default in core:net ).

          • Alternatively, use select , poll , or threads/coroutines to handle multiple connections concurrently.

  • Client :

    • dial

      • Establishes an active connection with a remote server.

      • Connects to an IP + port that is listening.

Security

  • Sender identification :

    • The remote IP and port can be read with conn.RemoteAddr() , but:

      • This can be forged in MITM attacks  if no encryption is present.

      • Even with IP/port, identity is not guaranteed  (e.g., NAT, spoofing, etc.).

    • In Godot:

      • The peer_id is sent along with the packet.

      • This would be extremely insecure if it weren’t for a supposed  verification between the sender peer_id and its address upon receiving a packet.

        • If the peer_id and address do not match the stored value in a hashmap, it is an attempted forgery.

      • This is completely insecure if the peer_id and address are forged simultaneously, in a spoofing attack.

      • In this case, encryption is required.

  • "Double listen"

    • "If a legitimate server is doing listen  on an IP:port, is it possible for another computer (a remote attacker) to also listen  on the same IP:port to intercept connections?"

    • Routers/Internet/DNS/etc will forward packets only to the host that has the corresponding public IP.

    • Example :

      • The legitimate server is at 192.168.1.100:8080 .

      • An attacker on another host (e.g., 192.168.1.250 ) tries to listen  also on 0.0.0.0:8080 .

      • Result:

        • The attacker only receives connections sent to its own local IP.

          • Connections sent to the legitimate server IP still go to 192.168.1.100, as IP routing determines.

    • Cases where the attack may work :

      • Involve network manipulation.

      • ARP spoofing / ARP poisoning (LAN)

        • The attacker impersonates the "server IP" on the local network.

        • If successful, LAN clients may redirect packets to them.

        • They can then listen  and receive connections destined for the legitimate IP.

      • DNS spoofing

        • The attacker tricks the client into thinking api.myserver.com  resolves to the attacker’s IP.

        • The client then connects to the attacker.

        • This attack depends on name resolution control, not listen  directly.

      • BGP hijacking (Internet level)

        • Much rarer and more complex.

        • The attacker announces false routes on the Internet backbone, intercepting traffic to the server’s IP.

        • Applicable only in highly sophisticated attacks.

Head-of-line Blocking

  • TCP ensures packets arrive in order .

  • If a packet is lost, all subsequent packets must wait for retransmission before delivery to the application. This creates a bottleneck, as correctly received packets get "stuck" until the lost packet is received.

  • Example :

    • You send 3 packets: P1 , P2 , P3 .

    • The receiver gets P1  and P3 , but P2  is lost.

    • TCP waits for retransmission of P2  before releasing P3  to the application.

    • Result: Unnecessary delay , because P3  could have been processed earlier.

  • This blocking is one reason TCP can be slow for games and other low-latency applications.

Redis

  • Redis is an in-memory storage system, often used as a cache or fast database to improve application performance, including multiplayer games. It is not a low-latency networking engine by itself, but can be used to coordinate game state data, server synchronization, and maintain real-time state variables.

  • Redis is an in-memory database that primarily uses the TCP  protocol for communication. It is designed for fast access and data caching, but is not a network transport solution for UDP. In network scenarios, Redis is usually combined with other technologies that handle UDP transport, but Redis itself is not a UDP transporter.

  • Advantages :

    • Very fast and ideal for temporary storage and quick access to data that needs to be shared in real time.

    • Redis offers replication, persistence, and clustering features, aiding scalability.

  • Disadvantages :

    • Does not provide a complete network solution, mostly used as support in networking and game server setups.

    • Working with Redis requires a network structure to consume the data (e.g., Photon , KCP , or others).

  • Pricing Model :

    • Redis is free and open-source under the BSD license for on-premise use.

    • Paid versions (like Redis Enterprise ) offer extra features, advanced scalability, and cloud support.

  • Ideal Scenarios :

    • Real-time games and applications needing fast cache and data sharing across multiple server instances.

UDP (User Datagram Protocol)

  • Communication protocol that sends data packets without establishing a reliable connection, making it faster but less secure than TCP.

  • Used in applications where speed is more important than reliability, such as video streaming or online gaming.

  • Godot:

    • "UDP is a simpler protocol, which only sends packets (and has no concept of a "connection"). No error correction makes it pretty quick (low latency), but packets may be lost along the way or received in the wrong order. Added to that, the MTU (maximum packet size) for UDP is generally low (only a few hundred bytes), so transmitting larger packets means splitting them, reorganizing them and retrying if a part fails.".

Closing ( close )

  • What it does :

    • No connection teardown occurs.

    • The local port is released, and kernel buffers are cleared.

Photon Realtime

  • Photon is a cloud networking solution designed specifically for game development. It supports UDP communication with tools tailored for games and real-time synchronization.

  • Photon Realtime is designed for multiplayer games focusing on low latency and scalability, primarily using UDP. It also supports TCP in some cases, but UDP is the common protocol due to its efficiency in gaming networks.

  • Advantages :

    • Scalable and optimized for online games, with integrated cloud infrastructure, reducing setup and maintenance overhead.

  • Disadvantages :

    • Costs increase with scale, as it follows a usage-based model. Customization is more limited.

  • Ideal Scenarios :

    • Multiplayer games needing a ready-to-use, highly scalable solution.

  • License :

    • Paid service based on subscriptions and volume of users/connections.

  • Pricing Model :

    • Offers free basic plans with limitations and paid plans as needed.

Photon Quantum

  • Photon Quantum is a network solution based on a deterministic engine, specifically for multiplayer games with advanced physics, such as fighting games and simulators. Instead of sending continuous state packets, Quantum sends user inputs, keeping physics synchronized for all players.

  • Photon Quantum uses UDP for network communication, essential for low latency and high performance in games requiring precise synchronization, particularly in deterministic physics games where response time is critical. Quantum is optimized to ensure input data is transmitted quickly with minimal delay.

  • Advantages :

    • Designed for highly synchronized games, Quantum handles games where state consistency is critical. Supports rollback and reconciliation, ideal for high-precision, delay-sensitive games.

  • Disadvantages :

    • Can be complex to implement for games without deterministic physics or synchronization needs. Paid solution with costs scaling with player count.

  • Pricing Model :

    • Photon Quantum is paid, with subscription and usage-based pricing. Free plans for testing and prototyping exist but with limitations.

  • Ideal Scenarios :

    • Multiplayer games with advanced physics and precision mechanics, like fighting games or simulations where all players must share the same game state.

Photon Server

  • Photon Server is a self-hosted solution for multiplayer game development. Unlike Photon Realtime (cloud service), Photon Server lets developers host their own server, reducing costs and offering greater control.

  • Photon Server supports UDP  and TCP . UDP is ideal for multiplayer games requiring rapid, frequent updates, such as FPS and MOBA games. Photon Server lets developers choose the protocol depending on game needs, with UDP being the low-latency choice.

  • Advantages :

    • Provides flexibility and scalability, allowing developers to control infrastructure and adapt it to game requirements. Ideal for developers preferring on-premise solutions for long-term cost reduction and network control.

  • Disadvantages :

    • Requires server setup and maintenance, which can be complex for developers without server infrastructure experience. Less scalable than a managed cloud solution.

  • Pricing Model :

    • Photon Server offers a free license for up to 100 CCU (concurrent users), ideal for development and prototyping. Beyond that, paid licenses are required, scaled by user count.

  • Ideal Scenarios :

    • Multiplayer games where developers want more control over infrastructure, especially useful for companies avoiding recurring cloud service costs.

Communication Protocols: UDP Networking Libraries

ENet

About

  • Free and open-source.

  • Designed for reliability over UDP with low latency.

  • Advantages :

    • Supports reliability and flow control over UDP.

    • Highly optimized for large-scale multiplayer games, especially for low-latency, real-time state communication.

    • Good scalability, supporting multiple clients.

    • Solid documentation and active community.

  • Disadvantages :

    • Slightly more complex implementation compared to other solutions due to fine-grained control.

  • License :

    • MIT License.

Packet Flags: ENet

  • Enet Github .

  • Documentation .

  • ENET_PACKET_FLAG_RELIABLE:

    • "packet must be received by the target peer and resend attempts should be made until the packet is delivered".

  • ENET_PACKET_FLAG_UNSEQUENCED:

    • "packet will not be sequenced with other packets. not supported for reliable packets".

  • ENET_PACKET_FLAG_UNRELIABLE_FRAGMENT:

    • "packet will be fragmented using unreliable (instead of reliable) sends if it exceeds the MTU".

Packet Flags: Godot

  • FLAG_RELIABLE

    • Mark the packet to be sent as reliable.

  • FLAG_UNSEQUENCED

    • Mark the packet to be sent unsequenced (unreliable).

  • FLAG_UNRELIABLE_FRAGMENT

    • Mark the packet to be sent unreliable even if the packet is too big and needs fragmentation (increasing the chance of it being dropped).

  • Godot Github . 

    switch (get_transfer_mode()) {
    case TRANSFER_MODE_UNRELIABLE: {
        packet_flags = ENET_PACKET_FLAG_UNSEQUENCED | ENET_PACKET_FLAG_UNRELIABLE_FRAGMENT;
        channel = SYSCH_UNRELIABLE;
    } break;
    case TRANSFER_MODE_UNRELIABLE_ORDERED: {
        packet_flags = ENET_PACKET_FLAG_UNRELIABLE_FRAGMENT;
        channel = SYSCH_UNRELIABLE;
    } break;
    case TRANSFER_MODE_RELIABLE: {
        packet_flags = ENET_PACKET_FLAG_RELIABLE;
        channel = SYSCH_RELIABLE;
    } break;
}
ENet Reliable

  • ENet is based on UDP, which does not guarantee packet delivery by default. To fix this, ENet implements its own reliability system:

  • ACKs (acknowledgment) and Retransmissions :

    • Each reliable packet has a unique sequence number.

    • When the receiver gets a packet, it sends an ACK confirming receipt.

    • If the sender does not receive an ACK in time, it retransmits the packet.

    • This continues until the packet is confirmed.

  • Dynamic Timeout :

    • ENet measures Round-Trip Time (RTT) and dynamically adjusts the retransmission time for lost packets.

    • Reduces unnecessary retransmissions and improves performance on unstable networks.

  • Example of ENet reliability :

    • Client sends P1, P2, P3

    • Server receives P1 and P3, but P2 is lost

    • Server sends ACKs for P1 and P3, but not P2

    • Client notices P2 was not acknowledged and retransmits it

    • Server receives P2 and sends an ACK

ENet Reliable vs TCP

  • TCP :

    • Reliable, guarantees ordered delivery without loss, retransmitting packets as needed. But it has more overhead and may have higher latency due to congestion control and acknowledgments.

  • ENet (reliable) :

    • ENet reliable is generally faster than TCP in most scenarios where it is supported.

    • Uses UDP as a base, but implements reliability with optional retransmissions and ordering. It is lighter and faster than TCP for games and low-latency networks, as it avoids some of TCP's restrictions, such as global congestion control and head-of-line blocking.

      • How does ENet avoid Head-of-Line Blocking?

        • ENet solves this problem in three main ways:

        • Multiple Channels

          • In ENet, packets can be sent on different channels within the same connection.

          • If a reliable packet is lost, only packets on that specific channel wait for retransmission.

          • Other channels continue sending packets unaffected.

          • Example:

            • Channel 0 sends P1, P2, P3

            • Channel 1 sends A1, A2, A3

            • If P2 (from Channel 0) is lost, only P3 waits. A1, A2, and A3 continue normally!

            • Result: Less delay, smoother communication

        • Retransmission without blocking other packets

          • ENet only retransmits lost packets at the channel level, without blocking the delivery of independent packets.

          • TCP, on the other hand, must guarantee global order, so a single lost packet can block everything.

        • Custom reliability control

          • In TCP, all packets are reliable and ordered.

          • In ENet, you can mix reliable and unreliable packets, choosing when to guarantee delivery and when to prioritize speed.

  • "When would you choose to use TCP then?"

    • Standard and compatible environment :

      • TCP is natively supported by any operating system and does not require an extra library like ENet.

    • Works well with firewalls and NAT :

      • Since it uses established connections, TCP has fewer problems with firewalls and NAT than UDP.

    • Transmission of large and continuous data :

      • For things like HTTP, downloads, file streaming, TCP is optimized to ensure everything arrives without loss and efficiently.

    • Less manual work :

      • With TCP, you don't need to manage retransmissions, congestion control, or implement your own reliability logic. It handles everything automatically.

  • Verdict :

    • ENet appears superior to TCP in almost all scenarios where it is supported, especially for online games, VoIP, and other applications requiring low latency.

    • Despite ENet's advantages, TCP still has its uses, mainly because it is native across all platforms and networks.

      • Universal compatibility – TCP works without additional libraries, while ENet requires implementation.

      • Works better with firewalls and NAT – Many networks block UDP (the base of ENet), while TCP is always allowed.

      • Use in non-interactive applications – For HTTP, downloads, file streaming, TCP works very well and is already optimized for that.

KCP (Fast and Reliable UDP Library)

  • KCP is a popular library for UDP focused on fast and reliable connections, widely used in games and network applications that require low latency.

  • Advantages :

    • Easy to integrate and offers customizable settings for flow control and packet loss recovery. It is a lightweight alternative to QUIC, maintaining good performance.

    • Excellent for high-latency and packet-loss environments, as it is designed to maintain reliability without sacrificing performance.

    • Widely used in games that require good scalability.

    • Simple implementation and easy to integrate.

  • Disadvantages :

    • Does not have built-in security and may require manual tuning for scalability and optimal traffic configuration.

    • Lacks advanced network control features like ENet, such as packet ordering or finely adjustable flow control.

  • Ideal Scenarios :

    • Multiplayer games and applications requiring low latency and reliable transmission over networks without built-in encryption.

    • Ideal for games needing reliable communication without the costs and complexity of a protocol like TCP. KCP is efficient in MMOs where latency is critical, but some packet loss can be tolerated.

QUIC (Quick UDP Internet Connections)

  • Developed by Google and now an IETF standard, QUIC is a UDP-based protocol that offers built-in reliability and security features (e.g., TLS encryption).

  • Advantages :

    • High scalability for multiple users, congestion control, and lost packet recovery, all with very low latency. It is one of the best options for reliable and secure data transmission over UDP.

  • Disadvantages :

    • More complex to implement than ENet and other libraries. Also, built-in security can require more computational resources compared to lighter solutions.

  • Ideal Scenarios :

    • Low-latency applications with high scalability, such as video streaming, multiplayer games, or secure web connections.

  • License :

    • Varies by library, but Quiche, for example, is licensed under the BSD license.

Reliable UDP (RUDP)

  • RUDP is a UDP extension that implements reliability, providing packet acknowledgment and retransmission while maintaining the simplicity of the protocol.

  • Advantages :

    • Offers more control over data transmission than pure UDP but with less overhead than TCP. Facilitates scalable implementation, as it has flow control and packet acknowledgment.

  • Disadvantages :

    • RUDP libraries can be more complex and vary in efficiency. Basic reliability may not be sufficient for data-intensive use cases.

  • Ideal Scenarios :

    • Applications that need more guaranteed delivery than pure UDP but do not require the full complexity and overhead of TCP.

  • License :

    • License varies depending on implementation, but many are MIT or BSD.

    • Several free implementations are available, especially on GitHub, although RUDP is more of a technique than a single library.

WebRTC (Web Real-Time Communication)

  • WebRTC enables real-time communication via UDP using data channels. It is widely used in browsers and supports NAT traversal features.

  • WebRTC .

  • Harder to implement than WebSockets and not necessary if you only need data exchange.

  • WebRTC explanation in 100 seconds .

    • Interesting.

  • WebRTC is a technology that enables real-time communication of audio, video, and data directly between browsers or devices without intermediate servers.

  • It is especially useful for peer-to-peer communication and is widely used in video and voice call applications.

  • It has low latency, making it ideal for real-time communication like video calls and fast file sharing.

  • Advantages :

    • Scalable and very useful for peer-to-peer connections. Provides built-in encryption and is free on local networks and easy to configure between browsers.

  • Disadvantages :

    • Focused on peer-to-peer connections, which limits use in centralized server architectures. NAT traversal overhead may add latency.

  • Ideal Scenarios :

    • Applications requiring P2P communication and moderate scalability, such as browser games or video/audio apps.

  • License :

    • Varies depending on implementation, but most browsers that support WebRTC use permissive licenses like BSD.

    • WebRTC is a free standard with open-source implementations in browsers and communication frameworks.

Comparison with WebSockets

  • Video call comparison between WebSockets and WebRTC .

    • Very cool.

  • Harder to implement than WebSockets and not necessary if you only need data exchange.

  • WebRTC uses technologies like UDP  to transmit data, which can result in lower latency compared to TCP (protocol used by WebSockets), especially on unstable networks.

Usage Examples

  • Video calls and Voice (VoIP).

  • File sharing.

  • Real-time games.

gRPC with UDP (non-native)

  • gRPC is a high-performance RPC library that usually works over HTTP/2 but can be adapted for UDP in some cases.

  • Advantages :

    • Since gRPC is widely adopted, adapting it to UDP in microservices can allow fast calls between services, with scalability in distributed systems.

  • Disadvantages :

    • Not natively UDP-based, so it requires adaptations. Scalability and performance may vary depending on the level of customization.

  • Ideal Scenarios :

    • Microservice environments requiring low latency but that can forgo full reliability and robust sequencing.

  • License :

    • Apache License 2.0.

    • gRPC is free and open-source, but adaptations are needed for use with UDP, which can be done without additional costs.

Lidgren.Network

  • Lidgren.Network is a network library aimed at games and is a reliable solution over UDP. It is written in C# and is popular for games requiring high performance and low latency.

  • Advantages :

    • Support for reliable connections and real-time communication.

    • Easy to integrate and highly optimized for game networks.

    • Designed for games with a high number of simultaneous users, with scalability capabilities.

  • Disadvantages :

    • Focused on C#, which may be a limitation if you are developing in another language.

  • Ideal Scenario :

    • Perfect for MMOs written in C#, especially on platforms like Unity, where Lidgren can be easily integrated with the game engine.

  • License :

    • MIT License

RakNet

  • A network library widely used in multiplayer games, supporting many players and features aimed at MMOs.

  • License :

    • Originally open-source under the MIT license, but later acquired by Oculus (Facebook), and the latest version is available under a commercial license.